Video and still-image cameras are increasingly becoming ubiquitous in our technologically-advanced society. In fact, cameras are now integrated into a broad range of electronic devices, such as mobile or cellular phones. It is quite common for mobile phone users to utilize an integrated camera on a regular basis, such as capturing an image or video of a contact and displaying the image of the contact when the contact calls the user of the mobile phone.
Furthermore, mobile terminals equipped with cameras allow users to capture images and video and then transmit the captured video and images to remote locations. Because of the size limitations placed on cameras and sensors that are part of mobile terminals, it is common for captured images to have poor quality. In particular, contrast and intensity levels are often set at levels that result in poor image quality.
Given the size, energy requirement, and processing limitations of mobile phones and other electronic devices, most integrated cameras will not have all features associated with traditional cameras, such as a flash. Therefore, some captured images may have impaired contrast. Thus, technological advances for contrast optimization have been developed.
When performing contrast optimization, methods generally attempt to utilize the dynamic range for a specific image and/or video segment as accurately as possible. Thus, the image pixels are processed to achieve optimal dynamic range utilization. Traditionally, the simplest solution has been the application of a predetermined filter to an image. This method is not very robust, however, because the image content defines the optimal target dynamic range.
Another prior solution has been to apply image adaptive filters. In one example, the image analysis process may resemble a human viewer interpretation of the image. Between these solutions, some features of the images can be selected for defining the target dynamic range and for basis of the image processing function calculation.
As readily known to those skilled in the art, different images traditionally require different solutions as they exhibit drastically different features. For example, FIGS. 1-5 provide examples of image histogram classes representative of images that traditionally require different processing methods. As presented throughout this application, the histograms illustrate the dynamic range of exemplary images with the x-axis representing the pixel values with the left most spectrum of the x-axis comprising the darkest pixel values and the right most spectrum comprising lighter pixel values. The y-axis represents the amount or qualitative level of that particular pixel value. A review of FIGS. 1-5 will further elaborate and explain on these concepts and introduce the reader to relative terms. Further, as explained in more detail below, despite the histograms having vastly different characteristics, they may all be effectively modified according to one or more embodiments of the invention.
FIG. 1 is an exemplary histogram of an image that utilizes the whole dynamic range almost equally. Traditionally, this kind of histogram is considered to illustrate a well contrasted image. However, the image represented by the histogram does have a slightly higher relative abundance “darker” pixel values than “brighter” ones as evidenced with histogram 102 being slightly more elevated on the left side than when compared with the right side.
FIG. 2 is an exemplary histogram of a low-contrast image. As seen with histogram 202, only part of dynamic range is utilized, specifically towards the middle of the dynamic range and absent from the right and left sides, indicating that dark and bright pixels are missing form the image, respectively. One prior method of contrast enhancement that may be applied to the image represented in FIG. 2 may be contrast enhancement by a fixed function, such as y=ax+b, where a=contrast gain and b=brightness addition. The parameters can be set manually, through a predetermined fixed contrast enhancement factor, or the values can be obtained somehow from the image (histogram), in which case the algorithm can be thought to be part of the next class. This solution may also be used for enhancement of the image represented in FIG. 3.
FIG. 3 is an exemplary histogram of another low-contrast image. Unlike FIG. 2, however, the histogram is not at the middle intensity level. Rather, as seen with histogram 302, the image has large bright area, but the darkest pixels are absent. Some pixel values are also overly saturated (i.e. burned to white). It is obvious to one skilled in the art that such a narrow histogram can be located around any intensity level and FIG. 3 is just but an example. While FIGS. 2 and 3 represent histograms of images that may be enhanced effectively with a fixed function, there are several other images, that may not effectively enhanced by fixed function, for example, due to driving some pixel values into saturation. For example, the image represented by the histogram in FIG. 1 and those presented in FIGS. 4-5 are not readily suited for this procedure. Further, in case of multiple intensity level areas, such as presented below in FIG. 5, only one of those areas of high intensity can be enhanced, while causing the other remaining ones are driven into saturation.
Histogram stretching is another method that may be utilized for contrast optimization. In histogram stretching, the parameters “a” and “b” are derived form the tails of the histogram. These are then are stretched to the ends of the dynamic range. The stretching can be further developed so that the different parts of the dynamic range are stretched differently, for example, as explained in Jain, Anil K., Fundamentals of Digital Image Processing, Prentice Hall, 1989, p. 235, incorporated herein by reference in its entirety. Similar to contrast enhancement by a fixed function, however, only the images represented by the histograms in FIGS. 2 and 3 may be optimized. Furthermore, there may be the problem of determining how to find the correct parameters for the stretching.
FIG. 4 is an exemplary histogram of another low-contrast image. Histogram 402 shows an exemplary histogram in which the whole dynamic range is utilized but there are substantially more dark pixels than bright pixels. Thus in the image represented by the histogram, there is a large dark area, which has quite low contrast. One skilled in the art will readily understand that the large area can be also bright, or having any other intensity level. Regarding FIG. 4 as presented, another method that may be utilized could be a gamma correction type of nonlinear function, such as expressed as y=xg, where “g” is the gamma value. The gamma value can be adaptive, e.g. <1 for histograms having most of values on the dark side and >1 for bright images. Similar to the contrast enhancement by a fixed function method, however, the adaptive gamma method can enhance only one of the different intensity level areas. Therefore, any others are impaired with reduced contrast. The adaptive gamma also requires class 1 or 2 algorithms for the processing of the complete dynamic range, because the gamma only modifies the relational proportion of different intensity levels, i.e. histogram (contrast) stretching is not achieved.
As introduced above, FIG. 5 is an exemplary histogram having multiple peaks with different intensity levels. As seen with histogram 502, there is a dark part and bright part in the image in which both have low contrast as seen with both peaks being relatively narrow. This kind of histogram is typical, for example, when an object is captured in front of bright background. In many practical cases, part of pixels are also saturated to black or white. One technique that may be applied to such an image may be histogram equalization to make a non-linear mapping function, such as expressed by y=f(x), that equalizes the image histogram. For example, the image pixel values are re-mapped so the approximate amount of pixels having each available intensity level is the same, such as explained in Jain, Anil K., Fundamentals of Digital Image Processing, Prentice Hall, 1989, pp. 241-242, incorporated herein by reference in its entirety. Therefore, the resulting histogram has a principal shape similar to histogram 102 one presented in the FIG. 1.
Histogram 302, presented in FIG. 3, may also be processed adequately with this method; however, the basic histogram equalization is too strong of an operation for the enhancement of natural images, because all histograms are made flat. Therefore, any intended variation is lost along with the natural appearance of the image. Gain limit modifications can be added, however, the balance between robustness and visible enhancement result is often too difficult to find.
One class of algorithms often utilized for contrast optimization utilizes spatially local information. For example, the signal may be divided into low-pass and high pass bands. The low-pass band presents the overall brightness of the spatial area. Gaining the high-pass band increases the contrast around that brightness level. Depending on the selection of frequency bands, at some point this algorithm class starts to resemble sharpening, which means enhancement of local contrast around edges. There exist multiple variations on how the overall brightness is estimated and how the contrast is processed. The problem, however, is that these algorithms tend to produce ringing type of artifacts, or are computationally more complex than the other classes. Moreover, given the size, energy requirements, and processing limitations of a mobile phones and other electronic devices, these methods are often too complex or otherwise too specific to select types of captured images or video
Hence, there is a need for efficient and effective methods and systems for performing contrast optimization for images having different characteristics. There is a further need for systems and methods that can perform contrast optimization with reduced processing and energy requirements.